Cyclical Local Structural Risk Minimization with Growing Neural Networks
نویسنده
چکیده
With that paper a new concept for learning from examples called Cyclical Local Structural Risk Minimization (CLSRM) minimizing a global risk by cyclical minimization of residual local risks is introduced. The idea is to increase the capacity of the learning machine cyclically only in those regions where the eeective loss is high and to do a stepwise local risk minimization, restricted to those regions. An example for the realization of the CLSRM principle is the TACOMA (TAsk Decomposition , COrrelation Measures and local Attention neurons) learning architecture. The algorithm generates a feed-forward network bottom up by cyclical insertion of cascaded hidden layers. The Output of a hidden unit is locally restricted with respect to the network input space using a new kind of activation function combining the local characteristic of radial basis functions with sigmoid functions. The insertion of such hidden units increases the capacity only locally and leads nally to a neural network with a capacity well adapted to the distribution of the training data. The performance of the algorithm is shown for classiication and function approximation benchmarks.
منابع مشابه
Boosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory
In this paper, several modifications to the Fuzzy ARTMAP neural network architecture are proposed for conducting classification in complex, possibly noisy, environments. The goal of these modifications is to improve upon the generalization performance of Fuzzy ART-based neural networks, such as Fuzzy ARTMAP, in these situations. One of the major difficulties of employing Fuzzy ARTMAP on such le...
متن کاملDyadic Classification Trees via Structural Risk Minimization
Classification trees are one of the most popular types of classifiers, with ease of implementation and interpretation being among their attractive features. Despite the widespread use of classification trees, theoretical analysis of their performance is scarce. In this paper, we show that a new family of classification trees, called dyadic classification trees (DCTs), are near optimal (in a min...
متن کاملComparison Study on Neural Networks in Damage Detection of Steel Truss Bridge
This paper presents the application of three main Artificial Neural Networks (ANNs) in damage detection of steel bridges. This method has the ability to indicate damage in structural elements due to a localized change of stiffness called damage zone. The changes in structural response is used to identify the states of structural damage. To circumvent the difficulty arising from the non-linear n...
متن کاملA New Topology for Artificial Higher Order Neural Networks: Polynomial Kernel Networks
AbstrAct Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feedforward neural network with a special polynomial activati...
متن کاملA Novel Extreme Learning Machine Based on Hybrid Kernel Function
Extreme learning machine is a new learning algorithm for the single hidden layer feedforward neural networks (SLFNs). ELM has been widely used in various fields and applications to overcome the slow training speed and over-fitting problems of the conventional neural network learning algorithms. ELM algorithm is based on the empirical risk minimization, without considering the structural risk an...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1996